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CLAIMS: 



1 M*/ \ ^ system for high dynamic range imaging comprising: 

2 / anVmage sensor comprising an array of light-sensing elements, the image sensor sensing 

3 an image of a scene Vnd providing corresponding pixel values representing light intensities impinging on 

4 respective ones of the light-sensing elements, the image sensor providing a saturated pixel value when 

5 the intensity of light impmging on a corresponding one of the light-sensing elements is greater than a 

6 first threshold level and providing a blackened pixel value when the intensity of light impinging on a 
p7 corresponding one of the lighnsensing elements is below a second threshold level, the image sensor 

W$ having a low dynamic range relative to the range of light intensities of the image of the scene; 

- \ 

a mask interposed between the scene and the array of light-sensing elements of the image 

rii \ 

ffj) sensor, the mask having a multiplicity of light-transmitting cells each controlling the exposure of a 

Dl respective one or more of the light-sensing elements to light from the scene, each of the light-sensing 

'i \ 

ftg elements having a corresponding exposure value indicative of the transparency of the cell through which 

f \ 

© light impinging on the light-sensing element passes; 

v 

14 a first memory storing exposure values corresponding to the light-sensing elements; and 

15 an image processor coupled to image sensor for receiving the pixel values provided 

16 thereby and coupled to the first memory for receiving the exposure values corresponding to the pixel 

1 7 values, the image processor comprising a normalizembr mapping the pixel values by a function of the 

18 exposure values to derive corresponding normalized pkel values. 
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1. 



2. The system according to jlaim 1, wherein the normalizer divides each of the pixel 
values with the exposure value corresponding to the light-sensing element receiving light intensity 
represented by the pixel value to derive coi responding normalized pixel values. 

3. The system according to claim 1, wherein the exposure values corresponding to the 
light-sensing elements are fixed and the in age processor further comprises an interpolator for 
interpolating the normalized pixel values t ) derive interpolated pixel values at respective positions of a 
second array overlapping the array of the light-sensing elements. 



4. The system according to claim 3, wherein the interpolator of the image processor 
interpolates the normalized pixel values, exlluding normalized pixel values corresponding to saturated 
pixel values or blackened pixel values. 

5. The system according to claim 3, wherein the interpolator of the image processor 
interpolates the normalized pixel values multiplied by respective weighting values. 



6. The system according to claim 5, wherein the weighting values are respective signal- 
to-noise ratios of the corresponding pixel values. 

7. The system according to claim 3, wherein the interpolator of the image processor 
applies an interpolation filter to the normalized pixel values. 
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8. The system according to claim 3, wherein the light-sensing elements of the array are 



disposed at respective intersections of a first 



values at respective intersections of a second 
extending in the vertical direction and in the 
one another. 



;rid defined by spaced intersecting perpendicular lines 



extending in a vertical direction and a horizontal direction, and the interpolator derives interpolated pixel 



grid defined by spaced intersecting perpendicular lines 
lorizontal direction, the first and second grids overlapping 



9. The system according to claim 8, wherein the interpolator of the image processor 
separately interpolates normalized pixel valu< ?s in each different region of the first grid, each region 
overlapping adjacent regions and encompassing a predetermined number of intersections of the first 
grid. 



aim 9, wherein the mask comprises repetitively disposed 



10. The system according to c 
identical groups of cells having different transparencies 



11. The system according to cljaim 10, wherein each of the repetitively disposed identical 
groups of cells of the mask consists of four neighboring cells having different transparencies, each cell 
controlling the light exposure of a different one of the light-sensing elements, and the first and second 
grids have identical dimensions and are displaced from one another by one-half grid position in the 
horizontal direction and one-half grid position in the vertical direction. 
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12. The system according tp claim 9, wherein the interpolator of the image processor 
derives a respective interpolated pixel value in each different region of the first grid by taking the 
average of all normalized pixel values inihe region which do not correspond to a saturated pixel value or 
a blackened pixel value. 



13. The system according to claim 1 further comprising a second memory storing a 
response function of the image sensor, wherein the image processor is coupled to the second memory for 
receiving the response function and further comprises a calibrator for converting the pixel values 
provided by the image sensor to linear response pixel values in accordance with the response function 



before the pixel values are normalized by 



14. The system according 



:he normalizes 



o claim 3 further comprising an output image memory 



coupled to the image processor for receiving and storing the interpolated pixel values. 



15. The system according 



claim 1 further comprising a first lens system located 
between the scene and the mask for projecting an image of the scene onto the mask and a second lens 
system for projecting the image of the scene as transmitted through the mask onto the array of light- 
sensing elements. 
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16. The system according 



;o claim 15, further comprising a light diffuser adjacent to the 



mask for removing directionality of light ocused by the first lens system. 



17. The system according 
solid state device image sensing device. 



18. The system accordin 
device light sensing array. 



19. The system according 



sensing array. 



20. The system accordir r 
sensing elements on the same substrate 



21. The system accordir 



material. 



o claim 1, wherein the array of light sensing elements is a 



to claim 17, wherein the solid state device is a charge-coupled 



to claim 18, wherein the solid state device is a CMOS light- 



g to claim 17, wherein the mask is integrated with the light- 



g to claim 1, wherein the mask is made of a nonlinear optical 



22. A system for high dy lamic range imaging comprising: 

risiii 



an image sensor compnsr 



an image of a scene and providing corres' 



g an array of light-sensing elements, the image sensor sensing 
bonding pixel values representing light intensities impinging on 
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respective ones of the light-sensing eleme its, the image sensor providing a saturated pixel value when 
the intensity of light impinging on a corresponding one of the light-sensing elements is greater than a 
first threshold level and providing a blacHened pixel value when the intensity of light impinging on a 
corresponding one of the light-sensing elements is below a second threshold level, the image sensor 
having a low dynamic range relative to tpe range of intensities of the image of the scene; 

a mask interposed between the scene and the array of light- sensing elements of the image 
sensor, the mask having a multiplicity of light-transmitting cells each controlling the exposure of a 



respective one or more of the light-sensi 



a mask controller coupled to the image sensor for receiving the pixel values provided 



sing elements to the light from the scene, each of the light-sensing 
elements having a corresponding exposi re value indicative of the transparency of the cells through 
which light impinging on the light-sensi ig element passes, the exposure value of each of the cells of the 
M mask being alterable by applying a respective exposure control signal to the mask; 

b 

ess. 

w 

I 

19 
20 
21 



thereby and for computing, and applying 



to the mask, exposure control signals for minimizing the 



number of saturated pixel values and blackened pixel values from the image sensor, and providing 
exposure values corresponding to the applied exposure control signals; and 

an image processor coupled to the image sensor for receiving the pixel values provided 
thereby, the image sensor comprising a ncrmalizer for mapping the pixel values by a function of the 
exposure values to derive corresponding normalized pixel values. 
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23. The system according jo claim 22, wherein the normalizer of the image processor 
divides each of the pixel values by the exposure value corresponding to the light-sensing element 



receiving light intensity represented by ths 



24. The system according 



processor is coupled to the exposure patte: 
the pixel values from the image sensor. 



pixel value to derive corresponding normalized pixel values. 



to claim 22, further comprising an exposure pattern memory 



coupled to the mask controller for receiving and storing the exposure values, and wherein the image 



memory for receiving the exposure values corresponding to 



He 



25. The system according tolclaim 22 further comprising an output image memory 
coupled to the normalizer of the image processor for receiving and storing the normalized pixel values. 



M 26. The system according to dlaim 24 further comprising: 

^2 a smoother coupled to the exposure pattern memory for receiving exposure values stored 

y| therein and applying a smoothing filter to the {exposure values to derive smoothed exposure values 

4 corresponding to the normalized pixel values; 

5 an exposure reapplicator coupl sd to the normalizer of the image processor for receiving 

6 the normalized pixel values and for multiplying each of the normalized pixel values with a 

7 corresponding smoothed exposure value to d< rive monitor image pixel values; and 

8 a monitor image generator coupled to the exposure reapplicator for providing the monitor 

9 image pixel values. 
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27. The system accordinglto claim 22, wherein the mask controller computes the 
exposure control signals to maximize thejsignal-to-noise ratio of the average pixel value provided by the 
image sensor. 

28. The system according to claim 22, wherein the mask controller computes the 
exposure control signals to maximize tne spatial smoothness of the exposure values corresponding to the 
light-sensing elements of the image sensor. 

29. The system according to claim 22, wherein the mask comprises a liquid crystal array. 



30. The system according to claim 22 further comprising a first lens system located 
between the scene and the mask for pre jecting an image of the scene onto the mask and a second lens 
system for projecting the image of the $cene as transmitted through the mask onto the array of light- 
sensing elements. 



3 1 . The system accord 



mask for removing directionality of li 



|h 



g to claim 30 further comprising a light diffiiser adjacent the 



ht focused onto the mask. 



32. The system accord; 
solid state image sensing device. 
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33. The system according to claim 32, wherein the solid state image sensing device is a 
charge coupled device light-sensing anjay. 

34. The system according to claim 32, wherein the solid state image sensing device is a 
CMOS light-sensing array. 

35. A system for high dynamic range imaging comprising: 
an image sensor comprising an array of light-sensing elements, the image sensor sensing 

an image of a scene and providing corresponding pixel values representing light intensities impinging on 



respective ones of the light-sensing e 
threshold level so as to cause the ima; 
light impinging on the light-sensing e j 
light-sensing element having a respeciji 
provide a blackened pixel value when 
below the respective second threshold 
range of light intensities of the image o 



ements, each light-sensing element having a respective first 
;e sensor to provide a saturated pixel value when the intensity of 
ement is greater than the respective first threshold level, each 
ve second threshold value so as to cause the image sensor to 
he intensity of light impinging on the light-sensing element is 
evel, the image sensor having a low dynamic range relative to the 
the scene, the light-sensing elements having respective spatially 



varying photosensitivies to incident ligqt and corresponding photosensitivity values indicative of the 
respective photosensitivities 

a first memory storing photosensitivity values corresponding to each of the light-sensing 

element; and 
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an image processor coup lid to the array of light-sensing elements for receiving the pixel 
values provided by the image sensor anc coupled to the first memory for receiving the photosensitivity 

; elements, the image processor comprising a normalizer for 
)f the photosensitivity values to derive corresponding 
ator for interpolating the normalized pixel values to derive 
bsitions of a second array overlapping the array of light-sensing 



values corresponding to the light-sensin 
mapping the pixel values by a function 
normalized pixel values, and an interpo 
interpolated pixel values at respective 
elements. 



36. The system according to claim 35, wherein the normalizer of the image processor 
maps the pixel values by a function of pe photosensitivity values by dividing each of the pixel values by 
the photosensitivity value corresponding to the light-sensing element receiving light intensity 
represented by the pixel value. 

37. The system according to claim 35, wherein the light-sensing elements of the image 
sensor are disposed at respective inters ections of a first grid defined by spaced intersecting perpendicular 
lines extending in a vertical direction znd a horizontal direction, and the positions of the interpolated 
pixel values are respective intersectior s of a second grid defined by spaced intersecting lines extending 
in the vertical direction and in the horizontal direction. 
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38. The system according 
separately interpolates normalized pixel 
overlapping adjacent regions and 
grid. 
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o claim 37, wherein the interpolator of the image processor 
values in each different region of the first grid, each region 
encompassing a predetermined number of intersections of the first 



39. The system according to claim 37, wherein the array of light-sensing elements of the 
image sensor comprises repetitively dispi >sed identical groups of light-sensing elements having different 
photosensitivities. 



40. The system accordin 
identical groups of light-sensing elemen 
different photosensitivities, and the first 
from one another by one-half grid positi 
vertical direction. 



to claim 39, wherein in each of the repetitively disposed 
s consists of four neighboring light-sensing elements having 
and second grids having identical dimensions and are displaced 
>n in the horizontal direction and one-half grid position in the 



41 . The system according to claim 38, wherein the interpolator of the image processor 
derives a respective interpolated pixel value in each different region of the first grid by taking the 
average of all normalized pixel values ir the region which do not correspond to a saturated pixel value or 
a blackened pixel value. 
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1 42. The system according to claim 35, wherein the image sensor has a respective 

2 response function to light impinging on eath of the light-sensing elements and the system further 

3 comprises a second memory storing the respective response functions of the image sensor, and wherein 

4 the image processor is coupled to the second memory for receiving the response functions and further 

5 comprises a calibrator for converting the pixel values provided by the image sensor to linear response 

6 pixel values in accordance with the respective response functions before the pixel values are normalized 

7 by the normalizes 

4 43. The system accordin i to claim 35, further comprising an output image memory 

p2 coupled to the image processor for receiving and storing the interpolated pixel values, 
m I 

: ~° I 
*p I 

a EJ 1 

stt 44. A method for high dvnamic range imaging comprising the steps of: 

s"" 3 I 

52 exposing an image sensoil comprising an array of light-sensing elements to an image of a 

aj i I 

1§ scene using a spatially varying exposure] the image sensor sensing the spatially varying exposure image 

# and providing corresponding pixel values representing light intensities impinging on respective ones of 

5 the light-sensing elements, the image senpor providing a saturated pixel value when the intensity of light 

6 impinging on a corresponding one of the light-sensing elements is greater than a first threshold level and 

7 providing a blackened pixel value when tie intensity of the light impinging on a corresponding one of 

8 the light-sensing elements is below a second threshold level, the image sensor having a low dynamic 

9 range relative to the range of light intensifies of the image of the scene; and 
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normalizing the pixel value^ provided by the image sensor with respect to the spatially 
varying exposure of the light-sensing elements to derive corresponding normalized pixel values. 

45. The method according Jto claim 44, wherein the step of exposing the image sensor 
using a spatially varying exposure includefe the step of using a mask having a multiplicity of light 
transmitting cells each controlling the expi >sure of a respective one or more of the light-sensing elements 
of the image sensor to light from the scene, each of the light-sensing element having a corresponding 

cy of the mask cell through which light impinging on the 
)f normalizing the pixel values provided by the image sensor 
includes the step of mapping the pixel vah.es by a function of the exposure values to derive 
corresponding normalized pixel values. 



exposure value indicative of the transparer 
light-sensing element passes, and the step 



46. The method accordiing 
a function of the exposure values comprise 



to claim 45, wherein the step of mapping the pixel values by 
dividing each of the pixel values with the exposure value 



corresponding to the light-sensing element receiving light intensity represented by the pixel value. 



47. The method according t$> 
light-sensing elements are fixed, and furthe 
values to derive interpolated pixel values at 
of the light-sensing elements. 
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claim 44, wherein the exposure values corresponding to the 
comprising the step of interpolating the normalized pixel 
respective positions of a second array overlapping the array 
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48. The method according to 
pixel values include the step of applying ar 



49. The method according 
pixel values includes the steps of separately 
region of the array of the light-sensing elements 
encompassing a predetermined number of t 



50. The method of claim 49 



pixel values in each different region of the 
the average of all normalized pixel values : 
pixel value or a blackened pixel value, to de 



5 1 . The method of claim 44 



provided by the image sensor according to 
response pixel values before the step 
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claim 44, wherein the step of interpolating the normalized 
interpolation filter to the normalized pixel values. 



claim 47, wherein the step of interpolating the normalized 
interpolating normalized pixel values in each different 

, each region overlapping adjacent regions and 
e light-sensing element positions. 



wherein the steps of separately interpolating normalized 
ajray of light-sensing elements each include the step of taking 
the region, except for ones that correspond to a saturated 
ive a respective one of the interpolated pixel values. 



urther comprising the step of calibrating the pixel values 
a response function of the image sensor to derive linear 
of normalizing the pixel values. 



52. The method according to 
interpolated pixel values in an output image 



claim 47 further comprising the step of storing the 



nemory. 
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53. The method of claim |5 further comprising the steps of projecting an image of the 
scene onto the mask, and projecting the i nage of the scene as transmitted through the mask onto the 
array of light-sensing elements. 

54. The method of claimJ53, wherein the step of projecting the image of the scene as 

afray of light-sensing elements includes projecting the image of 

ma$R and a light diffuser onto the array of light-sensing elements. 



transmitted through the mask onto the 
the scene as transmitted through the 



55. The method of clain 



sensing element is alterable by applying 

comprising the steps of: 

computing from the pixe 

for minimizing the number of saturated 
applying the computed 
providing exposure value^ 

in the normalizing step. 



56. The method accordin 



45, wherein the exposure value corresponding to each light- 
a respective exposure control signal to the mask, and further 



values provided by the image sensor exposure control signals 
rixel values and blackened pixel values from the image sensor; 
exposure control signals to the mask; and 

corresponding to the applied exposure control signals for use 



I to claim 55 further comprising the step of storing the 



normalized pixel values in an output ims ge memory. 
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57. The method according tfc> 
smoothing the exposure val 

applying a smoothing filter to the exposure 

the normalized pixel values; 

reapplying the smoothed exposure 

each normalized pixel value with a correspond] 



claim 55 further comprising the steps of: 
jes corresponding to the applied exposure control signals by 
values to derive smoothed exposure values corresponding to 



pixel values; and 

storing the monitor image pi 

58. The method of claim 55 



values to the normalized pixel values by multiplying 
ing smoothed exposure value to derive monitor image 



el values in a monitor image memory. 



wherein the step of computing exposure control signals 
comprises computing exposure control signals to maximize the signal-to-noise ratio of the average pixel 
value provided by the image sensor. 

59. The method according to claim 55, wherein the step of computing exposure control 
signals comprises computing exposure control signal to maximize the spatial smoothness of the 
corresponding exposure values. 



60. A method for high dynaituc 
exposing a frame of photograbhic 
multiplicity of light transmitting cells controlling 
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range imaging comprising the steps of: 
film to an image of a scene through a mask having a 
the exposures of respective regions of the film to light 
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from the scene, each of the respective regions of the film having a corresponding exposure value 
indicative of the transparency of the mas ted cell through which light impinging on the region passes; 
storing the exposure valm ;s corresponding to the respective regions of the film; 
processing the film to dc\ elop a masked image contained thereon or on a print thereof; 
scanning the processed film or a print thereof with an image scanner to derive pixel 

gions of the film or a print thereof; and 

ues from the scanner using the stored exposure values to derive 



values corresponding to the respective rp; 

normalizing the pixel va 



normalized pixel values. 



61 . The method accordi] i 



normalized pixel values to derive interp 



g to claim 60 further comprising the step of interpolating the 
elated pixel values. 



63. The method of claim 
corresponding to the respective regions 
corresponding to the respective regions 



62. The method according to claim 61 further comprising the step of storing the 
interpolated pixel values in an output iir age memory. 



60 further comprising the step of aligning the pixel values 
)f the film or a print thereof with the exposure values 
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64. The method acc Drding to claim 60 further comprising the step of calibrating the pixel 
values corresponding to the respect ve regions of the film or a print thereof in accordance with a 
response function of the film, the ir lage scanner and a printer, if any, to derive linearized pixel values 
before the step of normalizing the t ixel values. 
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